Sound library and method

ABSTRACT

At least one exemplary embodiment is directed to an electronic device configured to collect acoustic information or a method including the steps of collecting acoustic data by a microphone communicatively coupled to a mobile device, analyzing the acoustic data for a sound signature, tagging the sound signature with metadata, sending the sound signature with metadata to an acoustic database, associating sounds within the acoustic data with information, presenting the information on a map on the mobile device, and accessing at least audio from the acoustic database when a cursor is placed over a specific location on the map corresponding to captured ambient sounds from a geographic location and wherein acoustic information can be retrieved corresponding to the geographic location and for different selected periods of time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.12/560,074 filed on Sep. 15, 2009 which further claims the benefit ofU.S. provisional patent application No. 61/097,396 filed 16 Sep. 2008.The disclosure of the aforesaid applications are incorporated herein byreference in their entireties.

FIELD

The invention relates in general to sounds and methods of collectingsounds, and particularly though not exclusively, is related to creatinga library of sounds.

BACKGROUND

Acoustic sounds in the environment evolve with the progress ofcivilization and technology. Over time, people have adapted to theacoustic changes and learned to recognize sounds brought on by newtechnologies. The sounds of a large city differ as the mode oftransportation changes from walking to bicycles, cars, subways, andairplanes since each mode of transportation imparts its own uniqueacoustic sound. Sounds are a reflection of location, environment, andculture. For instance, the sound of the wind howling in an undisturbedcanyon in Australia or the sounds of traffic or a restaurant in TimesSquare in New York on New Years Eve are an unique acoustic snapshot of atime, place and event.

With all the information available today, there is not a singlesearchable database of naturally occurring and man-made sounds that isavailable to the world. The acoustic information that has been saved isin many types of media formats and stored in many locations making itdifficult to find and use. Moreover, the number of people preservingsounds is relatively small in relation to the acoustic information beingcreated on a daily basis. It is likely that with the rate of changeoccurring around us that sounds that were widely prevalent will nolonger be heard again. Preserving and making available acousticinformation of all types would be of substantial benefit to mankind.

SUMMARY

At least one exemplary embodiment is directed to a method of collectingacoustic information comprising the steps of: configuring a device toautomatically receive, collect, and send acoustic information; analyzinga first set of acoustic information to determine whether a trigger eventhas occurred; collecting a second set of acoustic information when thetrigger event has occurred; and providing the second set of acousticinformation to a database.

At least one exemplary embodiment is directed to a method of providingacoustic information with a map comprising the steps of: searching atopic; providing a map associated with the topic; searching an acousticdatabase for acoustic information related to the topic; and providingthe acoustic information related to the topic for playback with the map.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of present invention will become more fullyunderstood from the detailed description and the accompanying drawings,wherein:

FIG. 1 illustrates a diagram of a website as repository of acousticinformation in accordance with at least one exemplary embodiment;

FIG. 2 illustrates a diagram of a website and a user community formanaging and using a database in accordance with at least one exemplaryembodiment;

FIG. 3 illustrates a diagram of a user providing a sound signature tothe database of the website in accordance with at least one exemplaryembodiment;

FIG. 4 illustrates a diagram of using of a playback and managementenvironment in accordance with at least one exemplary embodiment;

FIG. 5 illustrates a diagram of a discussion webpage in accordance withat least one exemplary embodiment;

FIG. 6 illustrates a diagram of a database of sound pressure levels inaccordance with at least one exemplary embodiment;

FIG. 7 illustrates a diagram of an earpiece receiving a sound signaturefrom the website in accordance with at least one exemplary embodiment;

FIG. 8 illustrates a diagram of an earpiece partially sealing or sealingan ear of a user in accordance with at least one exemplary embodiment;

FIG. 9 illustrates a diagram of an earpiece in accordance with at leastone exemplary embodiment;

FIG. 10 illustrates a diagram of a communication device or earpiece forproviding sound signatures to a sound database in accordance with atleast one exemplary embodiment;

FIG. 11 illustrates a block diagram of a cell phone capturing a soundsignature and providing the sound signature to a database of sounds inaccordance with at least one exemplary embodiment;

FIGS. 12a-12c are related diagrams illustrating the use of soundpressure level as a trigger event configured to collect acousticinformation in accordance with at least one exemplary embodiment;

FIG. 13 illustrates a diagram of the use of geographic location as atrigger event configured to collect acoustic information in accordancewith at least one exemplary embodiment;

FIG. 14 illustrates a diagram of the use of time as a trigger eventconfigured to collect acoustic information in accordance with at leastone exemplary embodiment;

FIG. 15 illustrates a diagram of the use of sound signature detection asa trigger event configured to collect acoustic information in accordancewith at least one exemplary embodiment;

FIG. 15a illustrates triggering using an SPL value in a frequency band;

FIG. 15b illustrates a flow chart describing a triggering event usingperiodic signals;

FIGS. 15c-15k illustrates various spectrogram signatures thatillustrates the use of spectrograms for periodic detection in accordancewith at least one exemplary embodiment;

FIGS. 16a-16c illustrate a diagram of the use of sensor data as atrigger event configured to collect acoustic information in accordancewith at least one exemplary embodiment;

FIG. 17 illustrates a block diagram of downloading from a catalogue ofsound signatures 1210 in accordance with at least one exemplaryembodiment;

FIG. 18 illustrates a block diagram of an application where providingsound is informative and enhances a searching experience in accordancewith at least one exemplary embodiment;

FIG. 19 illustrates a block diagram of an application of threedimensional acoustical mapping in accordance with at least one exemplaryembodiment;

FIG. 20 illustrates a block diagram of an application for automaticallyproviding emergency information in accordance with at least oneexemplary embodiment;

FIG. 21 illustrates a block diagram of an application for detecting aburglary, intrusion, or serious situation in a building or home inaccordance with at least one exemplary embodiment; and

FIG. 22 illustrates a diagram of a website including a personal webpagefor socialization having an audio locker in accordance with at least oneexemplary embodiment.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The following description of exemplary embodiment(s) is merelyillustrative in nature and is in no way intended to limit the invention,its application, or uses.

Processes, techniques, apparatus, and materials as known by one ofordinary skill in the art may not be discussed in detail but areintended to be part of the enabling description where appropriate. Forexample specific computer code may not be listed for achieving each ofthe steps discussed, however one of ordinary skill would be able,without undo experimentation, to write such code given the enablingdisclosure herein. Such code is intended to fall within the scope of atleast one exemplary embodiment.

Additionally, the sizes of structures used in exemplary embodiments arenot limited by any discussion herein (e.g., the sizes of structures canbe macro (centimeter, meter, and size), micro (micro meter), nanometersize and smaller).

Notice that similar reference numerals and letters refer to similaritems in the following figures, and thus once an item is defined in onefigure, it may not be discussed or further defined in the followingfigures.

In all of the examples illustrated and discussed herein, any specificvalues, should be interpreted to be illustrative only and non-limiting.Thus, other examples of the exemplary embodiments could have differentvalues.

Although a method of sonic signature detection and identification isprovided herein, the scope herein should not be interpreted to belimited to the sonic signature description discussed. For example patentapplication Ser. No. 12/035,873, filed Feb. 22, 2008, titled “Method andDevice for Sound Detection and Audio Control”, and Ser. No. 11/966,457,filed Dec. 28, 2007, titled “Method and Device Configured for SoundSignature Detection”, both of which are incorporated herein in theirentirety, describe additional methods of sonic signature detection.

FIG. 1 is a diagram illustrating a system for capturing acousticinformation and storing the acoustic information in a repository inaccordance with at least one exemplary embodiment. The system 100 cancapture acoustic information around the world using one or morecommunication devices 106 (e.g., mobile and stationary communicationsystem (e.g., phones, computers)) communicatively coupled to one or moremicrophones 112 (e.g., Knowles FG microphones, balanced armature,diaphragm, and other acoustic recording and/or measuring devices),analyzing the acoustic information, determining what acousticinformation is relevant, providing relevant acoustic information to adatabase 110 (e.g., commercially available relational databases),organizing the acoustic information, and providing the acousticinformation for different applications. Note at least one non-limitingexample of the step of “determining what acoustic information isrelevant” can include determining whether the acoustic signal is abovethe noise floor or whether the acoustic signal falls within a topicselected (e.g., car horn, alarm) for example using sonic signaturemodels to identify whether the acoustic signal is the selected topic.

The system 100 leverages the World Wide Web, cellular networks, meshnetworks, and other networks used for communication as a path to gainaccess to a large number of people with socio-economic andgeographically diversity. By way of the system 100, people can collectsounds from their unique environment. This permits the efficient captureof various permutations of specific acoustic sounds in the environmentat various locations and at different times. Otherwise, it would be anoverwhelming effort for one entity, group of people or largeorganization to collect and manage such an enormous amount of acousticdata.

The system 100 permits manual or automatic recording and entering ofsounds into a database 110 for adding to the collection of capturedsounds. People can selectively capture sound at a particular time andlocation, and tag the sound to associate it with a particularexperience. Collectively the captured sounds can represent an ensemblesound experience or ‘acoustic snapshot’ at a particular location. Insuch regard, the system 100 permits a large segment of the population tocollect sounds 24 hours a day.

Communication devices 106 include a microphone configured to receivesound. Typically, people are mobile and carry their communication devicewith them. Collecting sounds using communication devices 106 willgenerate a tremendous amount of acoustic information as each personmoves from place to place during all times of the day and night.Communication devices 106 can also be adapted to automatically collectand send sounds to database 110 without user intervention and on acontinuous basis thereby creating database 110 of acoustic informationworld wide 24 hours a day. Thus, hundreds of millions or billions ofpeople could be collecting acoustic information for historical, social,scientific, medical, and business purposes.

In at least one exemplary embodiment, the collection of sounds isimplied in its broadest context to frequencies above and below humanhearing range and to the types of sounds being collected. The cry of ababy, a person snoring, the sound of a snake slithering on the ground,the noise of a city street, or rain in the desert, are just a fewexamples of sounds that can be recorded and stored and placed in contextto a location and a point in time.

In at least one exemplary embodiment, a catalog of distinct sounds canbe generated. Collected sounds will be organized to build an originalset of sounds representative of the distinct audible elements of ourlives. These sounds will be cataloged and cross-referenced so as to beindexable (e.g., vehicles, type of car, boat, ocean, beach, baby cry)and searchable on a number of levels. An example of a few categories forillustration purposes are residential sounds broken into sounds of thehome, sounds of the garage, sounds from the lawn and without thosecategories classifications for appliances, tools and machines that aretraditionally found there. Additional categories could be commercial(e.g. factories, call centers, construction Sites, offices, hotels,etc.), medical (e.g. dental offices, surgery rooms, hospitals, etc.),transportation (e.g. airplanes, buses, trains, subways, vehicles, etc.)to name but a few. In cataloging these sounds, it is also intended thatas much metadata as possible is captured and associated with the sound.There is also the aspect of collected sounds for use in socialization,which will be discussed further hereinbelow.

A further use can be military in nature. A littoral environment can bereal time acoustically mapped, tagged and sent via a securecommunication network. The tagged acoustic signals can be displayed on amap and electronically and/or manually analyzed to identify enemyactivity, location trends, ammunition usage, vehicle location andmovements, friendly location and movement, as well as other intelligenceuseful for the military planner.

FIG. 2 is a diagram illustrating a website 208 and a user community formanaging and using a database 212 in accordance with at least oneexemplary embodiment. Communication devices 200 automatically collectacoustic information 202 and upload the acoustic information through acommunication path 206 to a database 212. A communication path 206 canbe the internet, wired or wireless networks, satellite, that can coupleto database 212. Website 208 leverages a global member community to helpcatalog, provide information, and maintain the integrity of acousticinformation in the website. In at least one exemplary embodiment,communication devices can also download acoustic information fromdatabase 212 through communication path 206 for use on communicationdevices 200.

Website 208 allows participants to collaborate with others in capturing,identifying, cataloging, and formatting the acoustic informationreceived automatically through communication devices as well as manuallyprovided acoustic information. Global users 200 will be able to add,use, edit, delete, and modify the sound content within certain limits inwebsite 208.

The web pages of website 208 engage everyone to participate in adatabase 212 of acoustic information. Through an open web community,tens of thousands or millions of participants can join into the project.Thus, a database of sounds from around the world can be assembled in anefficient manner and accessible to everyone for present and futuregenerations. In at least one exemplary embodiment, users will have toregister once and login as indicated by block 210 to use the website.Once logged in, users will have access to the database within certainprivileges In at least one exemplary embodiment, user access to database212 can be based on his/her personal contributions or by paying foraccess.

The website 208 can format, catalog, and organize the sounds inaccordance with a common set of technology and organizationalguidelines. It can regulate and manage the collection and cataloging ofsounds via collaborative tools available in the community therebypermitting users to regulate the collection process and instill aspectsof quality control.

One aspect of website 208 is to catalog a diversity of sounds into acommon database. Each sound entered into database 212 is identified andcategorized so that it can be referenced and cross referenced on amultitude of different levels through their associated metadata. In atleast one exemplary embodiment, sound signatures are stored in database212. Sound signatures can also be stored as sound models. For exampleeach collection of a unique sound (e.g., horn, siren, bell, whistle,etc.) can be represented as a Gaussian Mixture Model (GMM). Theparameters of the GMM fully describe the sound in a pattern recognitionsense. The GMM can then be used to recognize new occurrences of thatunique sound, for example, a newly recorded horn sound. Having the GMMof a sound reduces the need for storing the entire acoustic soundwaveform in memory thereby reducing memory requirements when storingsuch a vast amount of information. The GMM can be used in searching,identifying, and comparing sounds in database 212. In at least oneexemplary embodiment, there will be at least one GMM for each sound(e.g. horn, whistle, snoring, etc.) stored in database 212.

FIG. 3 is a diagram illustrating a process for providing a soundsignature to the database of the website in accordance with at least oneexemplary embodiment. Sound signatures provided to the website arereviewed before incorporation into the database. In at least oneexemplary embodiment, the technical format 302 of the sound signature ischecked to ensure it meets the standards for incorporation. Meetingminimum technical standards and limiting the number of formats willallow the community to have access to quality recordings. In at leastone exemplary embodiment, technical format 302 can include programs thatcan convert non-compliant audio formats to a compliant format for thewebsite.

In at least one exemplary embodiment, the sound signature being providedis compared against others in the database in sound signature search304. The contributor or automatically generated metadata provides searchparameters associated with the sound signature that aids in thecomparison. In at least one exemplary embodiment, sound signatures forvarious sound categories can be represented by a Gaussian mixture model(GMM). A GMM would also be generated if only acoustic information wasprovided. Each GMM provides a model for the distribution of the featurestatistics for each sound signature in a multi-dimensional space. TheGMM can be used in the search or identification of an unknown sound. Thesound signature can also be compared for its uniqueness and technicalqualities against other sound signatures that reside in the database.Furthermore, the community of contributors to the website can play asignificant role in the decision process of the merits of the soundsignature once it has been placed in the website.

A response 306 is provided to the contributor if the decision is not toinclude the sound signature in the database. Response 306 can indicatethe various reasons why the sound signature is not used such as asimilar signature already exists, technical issues, or other factors notto include. A yes decision to incorporate the sound signature includesproviding metadata. In general, the metadata can information relevantand related to the sound signature. In at least one exemplaryembodiment, the metadata aids in searching, categorizing, and providinginformation on the sound (to help identify the sound signature or crossreference the sound signature with other sound signatures in thedatabase. Descriptions or data about the sound signature such as what,when, and where are part of the metadata provided. The metadata can growto a high level of sophistication as the database grows and the needs ofthe users and contributors are better understood.

As mentioned previously, a database or archive is being created ofsounds around the world. Knowledge of when the sound was recorded andwhere it was recorded is stored in the database along with the soundsignature. Thus, a component of the metadata can time stamp and geocode318. The time stamp is the time at which the sound signature was taken.This information can be taken and provided manually or providedautomatically as part of the recording process. The geocode is acoordinate measurement that provides a representation of the exact pointor location where the measurement is taken whether it is on the planetearth or somewhere in the universe. In at least one exemplaryembodiment, the device taking the sound measurement has GPS or thegeocode is provided using a separate global positioning system (GPS)device for providing the position where the sound signature was taken.Alternatively additional location methods can be used, for example celltower triangulation of signals sent. In at least one exemplaryembodiment, the recording device would be designed to provide time stampand geocode 318 automatically with the sound signature. A threedimensional acoustic map of the world can be generated including thechanges that occur over time with this database.

One example of metadata is information that juxtaposes sounds fromsimilar categories against others in those categories but with differentgeographic origination. Common devices but in different geographiclocations will have different sounds. Emergency or police sirens havethe same meaning (e.g. emergency vehicle or police are near and may beapproaching rapidly) but may sound radically different depending on thecountry of origin. This type of difference extends to a large number ofobjects such as vacuum cleaners, door bells, horns, etc. depending onthe country or continent (US, Europe, Asia, etc. . . . ). The metadataassociated with each sound signature will catalog all of the differentsounds into a common database, and then organize the sounds so that theycan be referenced and cross referenced on a multitude of differentlevels through their associated metadata.

In at least one exemplary embodiment, the website provides a form to befilled out to provide the appropriate metadata, this form can also beincorporated in a communication device and be automatically attached tosound signatures captured and being provided to the database. Soundsignature and metadata 308 is reviewed in a check information 310 step.The contributor or owner of the communication device can be notified tocorrect any errors found during check information 310. The soundsignature and metadata are stored in a step 312 into the database. Oncestored in the database global users and contributors 316 can review thesound signature and metadata 308 on website 314.

FIG. 4 is a diagram illustrating use of a playback and managementenvironment in accordance with at least one exemplary embodiment. Awebsite 402 uses collaborative editing of the content and structure. Alasting community is built and formed using this collaborative approachto the website. In at least one exemplary embodiment, an onlinecommunity helps catalogue, add information, add features, and maintainthe website.

As mentioned hereinabove, the sounds are be tagged with associatedmetadata that will describe what the sound is. An example of a capturedsound and the metadata associated therewith is a 2004 Lincoln Towncarhonking horn on the corner of 48^(th) and 6^(th) Avenue in NYC near theRockefeller Center at 2 pm on Monday, Jun. 16, 2008. The owner of thecommunication device may also provide descriptions to the metadata ofthe sound signature that exists in relation to the primary sound such asother vehicles in transit, pedestrian traffic, and policeman's whistle.Conversely, a comparison of sound signatures in database 408 may yield agood match and the associated metadata of the match such as horn, car,street, etc. could be attached to the new provided sound signature.

Website 402 environment is a tool to research, upload, catalog, listento and maintain a database of acoustic information. An author 404 orglobal users and contributors 410 can utilize a management environment406 that allows author 404 or global users and contributors 410 tomanage aspects of the sound signatures. In at least one exemplaryembodiment, author 404 or global users and contributors 410 usemanagement environment 406 to modify or remove an existing soundsignature, add new sound signatures, and add to or modify associatedmetadata stored within database of sound signatures 408. The addition ofnew tools that support all aspects of sound signature capture and usecan be incorporated into website 402. Similarly, new information about asound signature or how the sound signature can be used can be added byauthor 404 or global users and contributors 410. Needs driven by theutility of having a large database of sound signatures will encouragethe community to contribute to increase the functionality of thewebsite, grow the size of the database, add new features, and add newinformation.

Website 402 includes a playback environment 412. Playback environment412 allows for sounds to be played back through the website using commonaudio protocols. Playback environment 412 accesses database of soundsignatures 408 prompted by a user for playing back a selected soundsignature.

FIG. 5 is a diagram illustrating a discussion webpage in accordance withat least one exemplary embodiment. A website 506 includes database ofsound signatures 516. In general, users of website 506 may recordacoustic information that they believe is interesting and meritsdiscussion among the community of users. Website 506 includes adiscussion webpage 508 for posting acoustic information or soundsignatures of interest.

An author 502 or user with acoustic information accesses website 506.Author 502 wants to post the acoustic information in the discussionwebpage 508. The acoustic information is uploaded in a step 518 towebsite 506. The acoustic information is checked for both technical andcontent merits. The acoustic information is rejected and not posted ifthe criteria are not met. In at least one exemplary embodiment, author502 may be notified about the reasons for rejection allowing correctionof the faults. Although, acoustic information is discussed hereinabove,the meaning of acoustic information does not merely imply only sound. Ingeneral, the data or information provided by users will include acousticinformation but may include any other material. An example of additionalinformation along with the acoustic information is video information.

The acoustic information provided by author 502 is accepted if website506 criteria are met. The acoustic information is stored in database 516and is provided on the website in a post audio step 522. In at least oneexemplary embodiment, discussion webpage 508 includes a discussionthread 510 corresponding to the posted acoustic information that may bestarted by author 502 or perhaps someone else within the community.Global users and contributors 504, seeing discussion thread 510 canplayback the posted acoustic information using playback environment 512.Global users and contributors 504 can respond and discuss the postedacoustic information to begin or continue a conversation on a specifictheme. For example, a user could post a recorded sound and soundpressure level of a busy street. The discussion could take the form ofwhat is the loudest city in the world. Another example of a thread couldbe the impact of continuous noise on the health of a human being.

FIG. 6 is a diagram illustrating a database of sound pressure levels inaccordance with at least one exemplary embodiment. As mentionedpreviously, the world is ever changing; the sound and sound levelschange over time. There is no single database or catalog of differentsounds found throughout the world. Furthermore, there is very littleinformation available about sound pressure levels. Sound pressure levelshave changed drastically over time due to climatic, geographic,environmental, and natural conditions. Communication device 602automatically uploads acoustic information or sound signatures thatinclude sound pressure level (SPL) measurements. In general,communication device 602 is adapted for measuring SPLs. The acousticinformation is checked as discussed hereinabove. The acousticinformation is not used if it does not meet established criteria. Thesound signature is prepared for storing in database 608 by includinginformation about the sound pressure level with the sound signature.Metadata including time stamp and geocode 614 is used to categorize andcatalogue the sound signature and sound pressure level measurement.Furthermore, the metadata aids in a searching and providing informationon the sound signature and sound pressure level measurement. The soundsignature, sound pressure level, and metadata are then stored indatabase 608. The user community of website 604 then has access to theinformation.

The sound pressure level measurement is typically a measure of soundrelative to a reference value. For example, the threshold of humanhearing is a common reference value. Sounds pressure levels measuredabove the threshold of human hearing would be audible to an averagehuman being. Having a database of measured sound pressure levels aroundthe planet or the universe that is continuously updated would havesubstantial utility to the scientific, medical, government, businesses,and the individual. In at least one exemplary embodiment the quality ofthe sound pressure level measurement or any acoustic recording would bechecked for quality control. For example if a microphone type isidentified that did the recording, then a microphone database canreferenced to determine a microphone response function. The microphoneresponse function can be used to check whether the recorded signalexceeds operational range of the microphone. Additionally if calibrationinformation is available for the recording microphone, the calibrationinformation can be used to adjust any recording.

One example is disclosed in published reports on the long term exposureof sound to human health. It is well known that loud sounds can damagethe ear. Long term exposure to even moderate sound pressure levels candamage the ear or produce hearing or frequency loss. Moreover, the soundcontent can also play a role in a person's health. People have differentcoping mechanisms for dealing with sounds. Too much sound or too muchdiversity of sound can be difficult for the brain to process. Physicaleffects such as depression, sleeplessness, or other ailments can resultof sound exposure. Having a database of sound pressure levels, time,date, and geographical location would have substantial utility tomankind. For example, trends in sound pressure levels over timecorrelated with health related issues in a geographic area where theambient sound is of a high level could show the impact of sound tobehavior and health. In another example, the sound pressure levelpertaining to a restaurant could have utility by a user wanting to knowif a restaurant is noisy or is the atmosphere conducive to a romanticdate. In a further example, noise patterns within a city could be usedas a factor in the location of a corporate headquarters and in theconstruction specifications related to sound insulation.

FIG. 7 is a diagram of an earpiece system receiving a sound signaturefrom the website 706. The earpiece itself can seal or partially seal anear. It should be noted that the earpiece system can take many shapesand sizes. Earpiece or headset configurations such as circum-aural,supra-aural, behind the ear, concha, in-ear, are designs that seal orpartially seal the ear. Earpiece 702 includes at least one microphoneand one speaker. In at least one exemplary embodiment disclosedhereinbelow, earpiece 702 includes an ambient sound microphone, an earcanal microphone, and an ear canal receiver for respectively receivingambient sound, receiving sound in an ear canal of a user, and providingsound to the ear canal of the user.

Earpiece 702 is coupled to the website 706 via the World Wide Web 704 orother wired or wireless connection to receive a sound signature fromsound database 708. Earpiece 702 can be connected to website 706 througha wired or wireless connection. Earpiece 702 can have an interface thatdirectly connects to World Wide Web 704 or can use a medium such as apersonal computer or cell phone that is connected to the internet. Forexample, earpiece 702 can connect through a USB cable to the personalcomputer to download one or more sound signatures. The display of thepersonal computer would be used to navigate website 706. Similarly,earpiece 702 can connect to a cell phone or laptop through a wirelessconnection such as Bluetooth, Zigbee, Wimax, or UWB. One or more soundsignatures are found through searching sound database 708 using the webenvironment provided by website 706. The sound signatures are thenstored in memory in earpiece 702 for use in a variety of applicationswhich will be discussed further hereinbelow.

FIG. 8 is an illustration of an earpiece device, generally indicated asearpiece that partially seals or seals a user's ear canal 824 and isconstructed and operates in accordance with at least one exemplaryembodiment of the invention. As illustrated, the earpiece comprises anelectronic housing unit 800 and a sealing unit 808. The earpiece depictsan electro-acoustical assembly for an in-the-ear acoustic assembly, asit would typically be placed in an ear canal 824 of a user 830. Theearpiece can be an in the ear earpiece, behind the ear earpiece,receiver in the ear, partial-fit device, or any other suitable earpiecetype. The earpiece can partially or fully occlude ear canal 824, and canbe configured for use with users having healthy or abnormal auditoryfunctioning.

The earpiece includes an Ambient Sound Microphone (ASM) 820 to captureambient sound, an Ear Canal Receiver (ECR) 814 to deliver audio to anear canal 824, and an Ear Canal Microphone (ECM) 806 to capture andassess a sound exposure level within the ear canal 824. The earpiece canpartially or fully occlude the ear canal 824 to provide various degreesof acoustic isolation. In at least one exemplary embodiment, assembly isdesigned to be inserted into the user's ear canal 824, and to form anacoustic seal with the walls of the ear canal 824 at a location betweenthe entrance to the ear canal 824 and the tympanic membrane (or eardrum). In general, such a seal is typically achieved by means of a softand compliant housing of sealing unit 808.

Sealing unit 808 is an acoustic barrier having a first sidecorresponding to ear canal 824 and a second side corresponding to theambient environment. In at least one exemplary embodiment, sealing unit808 includes an ear canal microphone tube 810 and an ear canal receivertube 814. Sealing unit 808 creates a closed cavity of approximately 5 ccbetween the first side of sealing unit 808 and the tympanic membrane inear canal 824. As a result of this sealing, the ECR (speaker) 814 isable to generate a full range bass response when reproducing sounds forthe user. This seal also serves to significantly reduce the soundpressure level at the user's eardrum resulting from the sound field atthe entrance to the ear canal 824. This seal is also a basis for a soundisolating performance of the electro-acoustic assembly.

In at least one exemplary embodiment and in broader context, the secondside of sealing unit 808 corresponds to the earpiece, electronic housingunit 800, and ambient sound microphone 820 that is exposed to theambient environment. Ambient sound microphone 820 receives ambient soundfrom the ambient environment around the user.

Electronic housing unit 800 houses system components such as amicroprocessor 816, memory 804, battery 802, ECM 806, ASM 820, ECR, 814,and user interface 822. Microprocessor 816 (or processor 816) can be alogic circuit, a digital signal processor, controller, or the like forperforming calculations and operations for the earpiece. Microprocessor816 is operatively coupled to memory 804, ECM 806, ASM 820, ECR 814, anduser interface 820. A wire 818 provides an external connection to theearpiece. Battery 802 powers the circuits and transducers of theearpiece. Battery 802 can be a rechargeable or replaceable battery.

In at least one exemplary embodiment, electronic housing unit 800 isadjacent to sealing unit 808. Openings in electronic housing unit 800receive ECM tube 810 and ECR tube 812 to respectively couple to ECM 806and ECR 814. ECR tube 812 and ECM tube 810 acoustically couple signalsto and from ear canal 824. For example, ECR outputs an acoustic signalthrough ECR tube 812 and into ear canal 824 where it is received by thetympanic membrane of the user of the earpiece. Conversely, ECM 814receives an acoustic signal present in ear canal 824 though ECM tube810. All transducers shown can receive or transmit audio signals to aprocessor 816 that undertakes audio signal processing and provides atransceiver for audio via the wired (wire 818) or a wirelesscommunication path.

The earpiece can actively monitor a sound pressure level both inside andoutside an ear canal 824 and enhance spatial and timbral sound qualitywhile maintaining supervision to ensure safe sound reproduction levels.The earpiece in various embodiments can conduct listening tests, filtersounds in the environment, monitor warning sounds in the environment,present notification based on identified warning sounds, maintainconstant audio content to ambient sound levels, and filter sound inaccordance with a Personalized Hearing Level (PHL).

In at least one exemplary embodiment, the earpiece can download one ormore sound signatures through a wired or wireless interconnection to awebsite. For example, the earpiece is connected through a personalcomputer or a cell phone to the website. The desired sound signature isdownloaded to the earpiece. In general, a sound signature is a sound orsounds that the user wants the earpiece to identify. The earpiece hasone or more microphones for hearing sounds. ASM 820 is coupled to theambient environment of the user. Conversely, ECM 806 is coupled to earcanal 824 and is isolated from the ambient environment by sealing unit808.

In at least one exemplary embodiment, ASM 820 is providing acousticinformation from the ambient environment to processor 816. Processor 816analyses the acoustic information for a sound similar to the soundsignature. Once identified, the earpiece will provide a response to thesound based on the application. For example, the user of the earpiecewould like to hear or be aware of ambulances or police cars when wearingthe earpiece. Note that similar can be a comparison of FFTs in frequencybands, where the difference between the values of the sound is similarto a particular sonic signature if the standard deviation amongst allfrequency bands is below a threshold value (e.g., 3 dB). Additionally aGaussian mixture model can be used where the confidence level is betterthan a threshold value (e.g., 80%) that the sound is a particularlydetected sonic signature.

The user downloads sound signatures from the website (or a websiteassociated with the database of website) to the earpiece related toambulances and police car sirens and horns. While the user is using theearpiece, processor 816 of the earpiece analyses acoustic informationprovided by ASM 820 for sounds similar to the downloaded soundsignatures. Upon identifying an ambulance or police car horn or siren inthe ambient environment, the earpiece will notify the user that theambulance or police car is approaching. In a first exemplary embodiment,the earpiece will reduce music or telephone call (or the dominant sourceof sound being provided by the earpiece) and amplify the identifiedsignal (ambulance or police car) thereby notifying the user of theapproaching vehicle. In a second exemplary embodiment, the earpiece willtell the user (through a synthesized voice) that an ambulance or policecar is approaching including the direction of the vehicle. The earpiececan also provide the identified signal with the voice warning. Othervariations are possible.

Conversely, the earpiece can perform the opposite operation. Theearpiece can identify a signal similar to the sound signature and thenattenuate it before providing it through ECR 814. For example, the userof the earpiece is a gun enthusiast. The user downloads a soundsignature related to a gun shot. The earpiece upon identifying the soundof the gun shot would attenuate the portion of the acoustic informationprovided by ASM 820 similar to the sound signature of the gun shot whileallowing other signals to come through. Thus, the user could engage in aconversation at the gun range with the gun shot sounds attenuated whilepassing the conversation through the earpiece thereby protecting his earfrom the loud sounds in this environment and being able to hear theconversation with more clarity.

The earpiece can generate an Ear Canal Transfer Function (ECTF) to modelthe ear canal 824 using ECR 814 and ECM 806, as well as an Outer EarCanal Transfer function (OETF) using ASM 820. For instance, the ECR 814can deliver an impulse within the ear canal 824 and generate the ECTFvia cross correlation of the impulse with the impulse response of theear canal 824. The earpiece can also determine a sealing profile withthe user's ear to compensate for any leakage. It also includes a SoundPressure Level Dosimeter to estimate sound exposure and recovery times.This permits the earpiece to safely administer and monitor soundexposure to the ear.

FIG. 9, a block diagram of an earpiece in accordance with at least oneexemplary embodiment. A power supply 905 powers components of theearpiece including microprocessor/DSP 906 (or processor 906) and a datacommunication system 916. As illustrated, the earpiece can include theprocessor 906 operatively coupled through a data communication system916 to an ASM 910, an ECR 912, and ECM 908. Data communication system916 may include one or more Analog to Digital Converters and Digital toAnalog Converters (DAC). The processor 906 can utilize computingtechnologies such as a microprocessor, Application Specific IntegratedChip (ASIC), and/or digital signal processor (DSP) with associatedRandom Access Memory (RAM) 902 and Read Only Memory 904. Other memorytypes such as Flash, non-volatile memory, SRAM, DRAM or other liketechnologies can be used for storage with processor 906. The processor906 includes a clock 934 to record a time stamp

In general, a data communication system 916 is a communication pathwayto components of the earpiece and components external to the earpiece.The communication link can be wired or wireless. In at least oneexemplary embodiment, data communication system 916 is configured tocommunicate with ECM assembly 908, ASM assembly 910, visual display 918,and user control interface 914 of the earpiece. As shown, user controlinterface 914 can be wired or wirelessly connected. In at least oneexemplary embodiment, data communication system 916 is capable ofcommunication to devices exterior to the earpiece such as the user'smobile phone, a second earpiece 922, and a portable media player 928.Portable media player 928 can be controlled by a manual user control930.

The user's mobile phone includes a mobile phone communication system924. A microprocessor 926 is operatively coupled to mobile phonecommunication system 924. As illustrated multiple devices can bewirelessly connected to one another such as an earpiece 920 worn byanother person to the user's mobile phone. Similarly, the user's mobilephone can be connected to the data communication system 916 of earpiece901 as well as the second earpiece 922. This connection would allow oneor more people to listen and respond to a call on the user's mobilephone through their respective earpieces.

As illustrated, a data communication system 916 can include a voiceoperated control (VOX) module to provide voice control to one or moresubsystems, such as a voice recognition system, a voice dictationsystem, a voice recorder, or any other voice related processor. The VOXmodule can also serve as a switch to indicate to the subsystem apresence of spoken voice and a voice activity level of the spoken voice.The VOX can be a hardware component implemented by discrete or analogelectronic components or a software component. In one arrangement, theprocessor 906 can provide functionality of the VOX by way of software,such as program code, assembly language, or machine language.

The RAM 902 can also store program instructions for execution on theprocessor 906 as well as captured audio processing data. For instance,memory RAM 902 and ROM 904 can be off-chip and external to the processor906 and include a data buffer to temporarily capture the ambient soundand the internal sound, and a storage memory to save audio informationfrom the data buffer in a compressed format responsive to a directive bythe processor. The data buffer can be a circular buffer that temporarilystores audio sound at a current time point to a previous time point. Itshould also be noted that the data buffer can in one configurationreside on the processor 906 to provide high speed data access. Thestorage memory can be non-volatile memory such as SRAM to store capturedor compressed audio data. The non-volatile memory could also be used tostore sound signatures.

Data communication system 916 can include an audio interface operativelycoupled to the processor 906 and the VOX to receive audio content, forexample from portable media player 928, cell phone, or any othercommunication device, and deliver the audio content to the processor906. The processor 906 responsive to detecting voice operated eventsfrom the VOX can adjust the audio content delivered to the ear canal ofthe user of the earpiece. For instance, the processor 906 (or the VOX ofdata communication system 916) can lower a volume of the audio contentresponsive to detecting an event such as a sound signature fortransmitting the acute sound to the ear canal of the user. The processor906 by way of the ECM 908 can also actively monitor the sound exposurelevel inside the ear canal and adjust the audio to within a safe andsubjectively optimized listening level range based on voice operatingdecisions made by the VOX of data communication system 916.

The earpiece and data communication system 916 can further include atransceiver that can support singly or in combination any number ofwireless access technologies including without limitation Bluetooth™,Wireless Fidelity (WiFi), Worldwide Interoperability for MicrowaveAccess (WiMAX), and/or other short or long range communicationprotocols. The transceiver can also provide support for dynamicdownloading and uploading over-the-air to the earpiece. It should benoted also that next generation access technologies can also be appliedto the present disclosure.

Data communication system 916 can also include a location receiver 932that utilizes technology such as a GPS (Global Positioning System)receiver that can intercept satellite signals and therefrom determine alocation fix of the earpiece. GPS receiver 932 configured operably withprocessor 906 can generate a geocode corresponding to a location andlink the geocode to an event such as a recording or sound pressure levelmeasurement.

The power supply 905 can utilize common power management technologiessuch as replaceable batteries, supply regulation technologies, andcharging system technologies for supplying energy to the components ofthe earpiece and to facilitate portable applications. A motor (notshown) can be a single supply motor driver coupled to the power supply905 to improve sensory input via haptic vibration. As an example, theprocessor 906 can direct the motor to vibrate responsive to an action,such as a detection of a warning sound or an incoming voice call.

The earpiece can further represent a single operational device or afamily of devices configured in a master-slave arrangement, for example,a mobile device and an earpiece. In the latter embodiment, thecomponents of the earpiece can be reused in different form factors forthe master and slave devices.

FIG. 10 is a diagram of a communication device 1002 or earpiece 1010 forproviding audio content to database 1008 in accordance with at least oneexemplary embodiment. Collecting a large number of sounds from aroundthe world is a daunting task. As mentioned previously, no group orbusiness entity would have the ability to acoustically map the world ona continuous basis. In at least one exemplary embodiment, the collectionof acoustic information is achieved by mobilizing as many people aspossible by making it simple to capture and provide a sound or sounds todatabase 1008. Furthermore, acoustic information can be collectedmanually using communication device 1002 and more efficiently byautomating the process of collecting and sending without any humanintervention.

In general, a sound signature is a sound that is collected, modeled,identified, and stored. An example of a group of related soundsignatures are warning sounds such as an alarm (e.g., bell, emergencyvehicle, security system, etc.), siren (e.g., police car, ambulance,etc.), voice (e.g., “help, “stop”, “police”, etc.), or specific noisetype (e.g., breaking glass, gunshot etc.). On a global basis, the soundsignatures would be different depending on geographic region. Forexample, a police siren in Europe is different than the United States orAsia. The number of different sound signatures that can be collected areas diverse as sound itself, from a bell clanging to a person snoring, atrain, a garbage truck backing up, or the wind on Mars. In general,these sound signatures are collected by the process disclosed herein,all over the world, and at different times using a common communicationdevice adapted to automatically collect sound, analyze the sound,determine if a sound should be saved, tagging the sound signature withmetadata, and sending collected sound signatures to a database. In atleast one exemplary embodiment, a Gaussian Mixture Model (GMM) of thecaptured sound signature could also be generated and provided.

Continuing with the example hereinabove and referring to the circuitryof FIG. 9. In at least one exemplary embodiment, communication device1002 or earpiece 1010 having circuitry similar to FIG. 9 can store audiocontent or sound signatures in memory 902 for previously learned soundsfrom which the processor 906 refers to for detecting a similar sound. Inthe example above some of the stored sound signatures are warningsounds. The sound signatures can be resident in memory 902 or downloadedto communication device 1002 or earpiece 1010 via the data communicationsystem 916 during operation as needed. Upon detecting a sound signature,in this example a warning sound, the processor 906 can take an action orresponse to the recognized sound signature. In this example, processor906 can report a warning to the user via audio delivered from atransducer on communication device 1002 or earpiece 1010 (if it is beingused).

In general, communication device 1002 or earpiece 1010 can monitor theenvironment through a microphone of each device for a sound similar to astored sound signature. Each sound signature has certain identifiablefeatures that characterize the sound. These features can collectively bereferred to as a sound signature which can be used for recognizing thesound. As an example, the sound signature may include statisticalproperties or parametric properties of the sound. For example, a soundsignature can describe prominent frequencies with associated amplitudeand phase information. As another example, the sound signature cancontain principle components identifying the most recognizable featuresof a sound.

Referring to FIG. 10, communication device 1002 is a mobilecommunication device having a microphone configured to receive sound.Examples of a communication device 1002 are a phone, cell phone, PDA,portable computer, GPS system with microphone, automobile, satellitephone, two way radio, smart phone, and an earpiece to name but a few. Itis well known that these devices are used throughout the world withaccess on every continent. Having the ability to automatically, detectand store sounds of interest, attach key information about the storedsounds, and then upload them for storage in a database of sounds withoutinterfering with the normal use of the device, mobilizes potentiallybillions of people for the collection of sounds. These sounds can thenbe used to the benefit of man for general knowledge, historicalpurposes, mapping, scientific, medical, and business to name a few thatwould be impossible to collect by other means.

Communication device 1002 and earpiece 1010 includes circuitry asdisclosed in FIG. 9 or is adapted to perform as disclosed hereinbelow.Communication device 1002 and earpiece 1010 is operably configured forrecording sound and measuring sound pressure levels. In at least oneexemplary embodiment, acoustic information received from microphone ofcommunication device 1002 or earpiece 1010 is placed in a buffer ormemory where it can be analyzed. The buffer is temporary storage andallows the continuous analysis of sound received from the microphone. Ifnothing of interest occurs, the sound in the buffer is discarded and thenext piece of sound information in the buffer analyzed.

A trigger event, which will be discussed in more detail in the nextfigure, is the event or decision that initiates the collection orcapture of audio content (or related acoustic information (example—soundpressure level of audio content) and sending of data to database 1008.In one exemplary embodiment, the audio content is stored in memory ofcommunication device 1002 or earpiece 1010 where it can be processedfurther and sent at a later time. Alternately, the audio content couldbe sent immediately to database 1008. The trigger event can be underuser control or controlled remotely through website 1006 and database1008.

Metadata is generated that can be used to identify aspects of the audiocontent, for example the trigger event (sound pressure level above acertain threshold), a time stamp of when the sound was recorded, or ageocode providing the location where the sound was recorded. Acommunication path 1004 can be a wired or wireless connection betweencommunication device 1002 or earpiece 1010 for transmitting the audiocontent, related information, and metadata. It should be noted that theaudio content is not always sent. For example, sound pressure level hasutility in mapping noise levels in different geographic areas/times andmay be sent with metadata (but the information intensive audio contentis not sent). In at least one exemplary embodiment, communication device1002 or earpiece 1010 automatically connects to a website 1006, serversystem, or database 1008 to upload the information. The audio content isuploaded where it may be reviewed further before being stored intodatabase 1008.

As mentioned hereinabove, another measurement that can be taken bycommunication device 1002 or earpiece 1010 is sound pressure level(SPL). The sound pressure level measurement can be taken and sent byitself (with associated metadata) or in combination with audio content.The sound pressure level (SPL) is measured from the microphone signalthrough an analog circuit or more typically after the microphone signalis converted to digital and calculated using digital processing beforeany audio processing (such as automatic gain control, equalization,etc.) occurs within communication device 1002 and earpiece 1010.

It should be noted that acoustic information can be recorded andprovided manually by the user of communication device 1002 and earpiece1008. If a manual process were relied on, it is likely that many or mostsounds would be missed and that only a segment of the potential soundsignature providers would participate. In at least one exemplaryembodiment, communication device 1002 or earpiece 1008 is always on,analyzing sound continuously, and storing audio content of interest.Performing the process automatically does not rely on the user of thecommunication device to provide information which opens a path for largenumbers of acoustic information to be provided continuously. An acousticmap of the world could be generated 24 hours a day because of the largenumber of devices spread over all geographic regions of the world.

In at least one exemplary embodiment, the user of communication device1002 or earpiece 1010 can manually add metadata that describes the eventto aid searching and categorizing a collected sound. For example, theuser can provide information that the recording is a fire truck sirenNew York City that is attached with the time stamp and geocode (which isautomatically tagged with the acoustic information). In at least oneexemplary embodiment, communication between the user of communicationdevice 1002 or earpiece 1010 can occur through communication path 1004requesting further information in a voice, text, or automated fashion.For example, after automatically sending sound signatures they arereviewed for format and in context to what is currently stored indatabase 1008. Further communication between could take place to edit,identify, describe, and format the provided captured sounds. Thecommunication could also occur at a more convenient time or venue(example at home through a personal computer) to further place it incondition to permanently store in the database 1008. It should be notedthat video information that includes audio information can also beprovided in similar fashion as disclosed hereinabove. The audioinformation can be removed from the video and used for database 1008.

Earpiece 1010 and communication device 1002 can be operably coupledtogether. A priority could be set up such that earpiece 1010 is theprimary recorder of sound 1012. Thus giving the collected informationgives the perspective of what the user hears. Earpiece 1010 can be usedwith other devices for example a portable media player. Earpiece 1010would collect, measure SPL, and tag metadata creating a queue of soundsthat are uploaded when a communication path 1004 is enabled. Thus, acommon device has been adapted for automatically capturing and storingsound information, measuring SPL, adding metadata including a time stampand geocode, and uploading the acoustic information to a databasethereby enabling the broadest number of people across the largestgeographic area for sound collection on a continuous basis.

FIG. 11 is a block diagram illustrating a communication device 1104collecting or capturing acoustic information and providing the sounds inaccordance with at least one exemplary embodiment. Communication device1104 includes at least one microphone that is enabled configured toreceive acoustic information. In at least one exemplary embodiment,communication device 1104 can have multiple microphones for averagingsound pressure level measurements. Moreover, people do not always keeptheir phone out in the open but have them on or in holsters and pockets.Multiple microphones increase the probability when automaticallycollecting audio content that an unobstructed microphone is availableconfigured to receive the information.

As mentioned hereinabove, the acoustic information is stored in acircular buffer in device 1104 and continuously analyzed. For example,the acoustic information in half the buffer is analyzed while newacoustic information is loaded into the remaining half of the circularbuffer. If a trigger event occurs where a sound of interest is detected,the acoustic information is stored in memory that is more permanent andnot written over in the communication device 1104 until it is providedto a database configured to collect the acoustic information.

A trigger event initiates the saving of acoustic information orinformation related to the sound that triggered the event for uploadingto a sound database 1118. A trigger event can take many forms. A soundpressure level (SPL) exceeding a certain threshold value could triggercollecting acoustic information. SPL triggered events would provideuseful information for mapping SPLs throughout an area and to identifyplaces where noise may be excessive. Alternately, a delta change insound pressure level could trigger the storing of the sound in thebuffer. For example, a spike or jump in sound pressure level that issubstantially higher than the ambient SPL could trigger the storing ofthe sound in the buffer (for example a gun shot or explosion).

Time is another example of a trigger event. Communication device 1104could receive and store acoustic information periodically. Periodicsampling could be used to map sound exposure of the user ofcommunication device 1104 which will vary depending on the time of dayas well as time of the year.

Geographic location is a further example of a trigger event.Communication device 1104 having a GPS receiver can identify a locationwith great accuracy. Communication device 1104 could trigger storing ofacoustic information for a specific location. For example, the user ofcommunication device 1104 has a trigger event that stores acousticinformation when visiting his/her doctor thereby keeping a medicalhistory of the visit. The acoustic information in this example does nothave to go to sound database 1118 for general use but could go to apersonally managed and secure “sound locker” that is owned by the userof communication device 1104.

Another example of a trigger event is sound signature detection. In atleast one exemplary embodiment, communication device 1104 includes orhas access to sound signatures. Communication device 1104 uses the soundsignatures to identify similar sounds. The acoustic information iscollected once a similar sound is identified from the sound signaturesavailable to device 1104. For example, sound database 1118 is collectingsounds related to warning sounds. Database 1118 can make available soundsignatures related to warning sounds to mobile communication deviceswhen a communication path is opened to the devices. Thus, a focus isplaced on collecting specific sounds automatically without requiringhuman intervention although both the collection and trigger event couldbe performed or entered in a manual process under user control. Othertrigger events such as voice activation and sensor data (movement,bio-information, atmospheric data, visual, substance detection, odor,etc.) are examples of events that are used to collect acousticinformation.

In at least one exemplary embodiment, a sound signature comprises aGaussian Mixture Model (GMM). In general, A/D converters in device 1104convert the analog sound signal to a digital sound signal. The processorchops up the digital sound signal into frames, for example, at fs=8000Hz a 20 ms frame of data is 160 samples; each sample is represented by16 or 32 bits (e.g., it is quantized). Thus a 1 second recorded anacoustic sound wave will be represented by 50 frames (20 ms*50=1 sec).

Each frame is then extracted for features as known in the art. Thefeatures can be Fourier series coefficients (FFT based) that represent afrequency decomposition of the frame; the features can be mel-cepstralor LPC coefficients that represent a spectral envelope decomposition.The features can be DCT, KLT, PCA, or any other feature set. Notably,the features provide an efficient form of data compression to reduce thedimensionality of the input space. (For instance, instead of using all160 samples (16 bits/sample) to represent the frame, the mel-cepstralonly requires 10-14 samples (16 bits/sample) to represent the sameframe).

The features are then used to train a GMM. There is a single GMM foreach sound signature. A sound signature consists of the features of theacoustic sound wave for instance, sequential frames of mel-cepstralcoefficients for a recorded acoustic sound wave. The training causes theGMM to learn the statistics of the features collectively called thefeature set. More specifically, the GMM represents statisticalparameters of the features set, in particular, the mean, covariance, andprobability weights of the features set. So, the GMM characterizes thestatistics of the features set using these parameters (mean, covariance,weights). So, for example, during training, the mean of the feature set(e.g., 10-14 cepstral coeffs for each frame) is collectively determined(e.g., average the cepstral coefficients over all frames), thecovariance of the features set is determined (calculate second moment ofthe cepstral coefficients over all frames), and the probabilities aredetermined (e.g., determine the frequency of occurrence of the frameswith respect to the number of GMM cluster centers (e.g., mean vectors).

Once the GMM has been trained the mean, covariances, and weights fullydescribe the sound signature. That is, these parameters fully specifythe GMM for modeling—in a pattern recognition sense—the sound signature.These parameters are then stored in memory for the particular GMM andused for reference when attempting to recognize sound signatures inambient sound. Thus, instead of saving the entire acoustic soundwaveform in memory for comparative purposes, the features are firstextracted, and then a GMM is created to model the statistics of thefeatures. The GMM is then used to recognize sound signatures of the samesound source during ambient sound monitoring.

In general, in database 1118 there will be as many GMMs as there aresounds that have been collected. Thus, there will be a GMM for a siren,another GMM for a horn, another GMM for a snore, etc. Then when the GMMsparameters (means, covars, weights) are all stored in memory, they canbe later retrieved from memory to determine if a new sound signatureprovided by a communication device is one of the learned sounds alreadystored on database 1118.

Similarly, communication device 1104 having GMMs of recognized soundsignatures can compare sounds received from the microphone of device1104 to the sound signatures. More specifically, the process ofrecognizing a new sound signature consists of the same front-end featureextraction steps; that are, generating a feature set (e.g., frames ofmel-cepstral coeffs). This features set is then passed to each GMM forassessing whether the parameters (mean, covar, weights) of the GMM are abest match to the new feature set. More specifically, the statistics ofthe new features set are compared in a maximum likelihood (ML) manner tothe parameters of each GMM. The new feature set is effectively mapped tothe parameters (means, covars, weights) of each GMM to determine whichGMM is most likely to represent the features in a maximum likelihoodsense, for instance employing principles of minimum distortion, smallestL-norm distance, and ML estimators. Each GMM produces a probability (forinstance between 0 and 1) representing a match to the new soundsignature; for instance, a GMM with a 0.1 probability output says thereis a 10% probability that the sound signature corresponds to the soundassociated with the GMM (‘whistle GMM). A GMM with a 0.9 probabilityoutput says there is a 90% probability that the sound signaturecorresponds to the sound associated with the GMM (‘siren GMM). Thus, thecriteria for adding a sound signature to the database can vary. Forexample, communication device 1104 providing a sound signature with lowmatch probability with sound signatures in database 1118 may be storedbecause it is unique. Conversely, a new sound signature having arelatively high match probability might be kept because of other factorssuch as location, time, or database 1118 is collecting sounds of thattype.

Continuing with the block diagram of FIG. 11, sound information storedin the buffer of device 1104 is analyzed and a trigger event determinesthat the received sound should be stored. Sound 1102 can be converted toan appropriate form such as a GMM for use by database 1118 or the soundinformation itself can be stored in memory of device 1104. The soundpressure level (SPL) of the acoustic information is measured orcalculated in a step measure SPL 1108. The measurement of a soundpressure level can be done in conjunction with a collected sound orindependently (where only the SPL is kept and the sound information isdiscarded).

Metadata 1110 corresponding to the captured or collected sound isattached to the sound information. Additionally, the user of device 1104can add further information prior to it being sent or later incommunication with database 1118. For example, the user can manuallyenter metadata 1110 via a keyboard to a metadata table or can be vocaldescription in an attached audio stream. Metadata 1110 includes a timestamp and a geocode corresponding to the sound signature. In at leastone exemplary embodiment, if the sound information is not converted to aGMM by communication device 1104 then a GMM will be generated of thesound when received by database 1118.

In at least one exemplary embodiment, the sound, sound pressure level,and metadata are stored in memory 1112 that resides on communicationdevice 1104. A queue of sounds 1114 can be stored in memory 1112 foruploading at an appropriate time. The user can initiate uploading of thequeue of sounds 1114 to sound database 1118 when a communication path iscompleted. In at least one exemplary embodiment, device 1104 canautomatically connect to servers in sound database 1118 and upload queueof sound 1114 when a communication path is enabled without any manualintervention by the user of device 1104 thereby making it a transparentprocess that does not interfere with normal operation of device 1104.Although stored on database of sounds 1118, there may be an iterativeprocess to determine if the acoustic information is in the correctformat or are unique enough to be permanently stored.

FIGS. 12a-12c are related diagrams illustrating the use of soundpressure level as a trigger event configured to collect acousticinformation in accordance with at least one exemplary embodiment. FIGS.12a-12c relate to the trigger event illustrated in FIG. 11. Referring toFIG. 11, a communication device 1104 is receiving acoustic informationthrough the microphone of the device. In at least one exemplaryembodiment, the acoustic information is stored in a buffer ofcommunication device for analysis. The analysis includes looking for atrigger event related to sound pressure level to initiate collection ofthe acoustic information.

Referring to FIG. 12a , a graph of sound pressure level versus time isshown that is calculated by the communication device from receivedacoustic information. A trigger event occurs when the sound pressurelevel of the acoustic information exceeds sound pressure level threshold1124. For example, information on high noise areas is being collected.Setting sound pressure level threshold 1124 at 70 dB would collectinformation in areas having a sound pressure level that exceeds 70 dBcould produce hearing loss if exposed to this level of ambient sound.The harm to the ear is related to the sound pressure level and period oftime exposed to the noise. Collecting a large number of data pointswould allow the mapping of acoustic information over a three dimensionalregion and over time. This information would have a variety of uses oneof which is identifying when and where high noise occurs in a city.

In at least one exemplary embodiment, the trigger event initiates thecollection of the acoustic information during a time period in which thetrigger event occurs. For example in FIG. 12a , a first trigger event1120 occurs where sound pressure level threshold 1124 is exceeded duringa time period t₁-t₂ as indicated by the dashed line. Once triggered, theacoustic information, as shown in FIG. 12b , during time period t₁-t₂ iscollected for sending to a database. The acoustic information can bestored in memory of the communication device in it's entirety orconverted to a more compact form, modeled, characterized, and providedwith metadata depending on the collection needs. Included in themetadata can the time information (t₁-t₂) and location informationassociated with the acoustic information. Referring to FIG. 12c , theposition of the communication device indicated in x, y, and zcoordinates versus time is indicated in the graph. The geographicinformation is provided with the metadata to identify where the acousticinformation was collected. The position of the communication device canbe static or moving over time which the information will indicate.

Similarly, a second trigger event 1122 is illustrated in FIG. 12a .Sound pressure level 1124 is exceeded during the time period t₂-t₃ asindicated by trigger event 1122 and the corresponding dashed line. Theacoustic information, as shown in FIG. 12b , during time period t₂-t₃ iscollected. Metadata including the time information and geographiclocation information will be attached with the acoustic information asdescribed herein above.

In the example, a trigger event occurs anytime the sound pressure levelthreshold 1124 is exceeded. The trigger event could be modified in otherways. Examples of sound pressure level trigger events are the soundpressure level being above threshold 1124 for a predetermined timeperiod, average sound pressure level above a predetermined threshold(over the time period), or a delta change in sound pressure level abovea predetermined amount to name but a few. Furthermore, the acousticinformation collected is not limited to the time period in which thetrigger event occurs. The amount of acoustic information collected canbe varied based on need. For example, collecting only acousticinformation that exceeds sound pressure level threshold 1124.Conversely, a trigger event could collect the acoustic information fromthe previous, current, and next time period.

FIG. 13 is a diagram illustrating the use of geographic location as atrigger event configured to collect acoustic information. Acommunication device 1131 includes a GPS receiver 1134 for providingpositioning information. One or more geographic locations are stored incommunication device 1131 or communication device 1131 has access togeographic locations for initiating a trigger event. Communicationdevice 1131 compares a current geographic location with the one or moregeographic locations using information from the GPS receiver 1134. Atrigger event configured to collect acoustic information occurs when thecurrent geographic location falls within one of the stored geographiclocations.

A geographic area 1136 is represented by a coordinate range of x±Δx,y±Δy, and z±Δz. Although three dimensions are indicated in the example,a single dimension or two dimensions could be used to define an area ofinterest. For example, using x±Δx and y±Δy could be used to identify arestaurant the z coordinate would not be necessary. People going to therestaurant at different times of the day (each having theircommunication device triggering off of the restaurant coordinates) wouldcollect acoustic information on the restaurant. For example, soundpressure level (SPL) measurements could be taken by a number ofdifferent people. The SPL measurements could be used to create a map ofthe restaurant indicating noise levels at different times of the day andin different locations of the restaurant. The information would utilityto users of the restaurant, for example, wanting to know when the noiselevels are low and where the most intimate (quiet) area of therestaurant is.

In at least one exemplary embodiment, a GPS receiver 1134 periodicallyprovides information on the location of a communication device 1131. Theperiodicity of the received GPS information is represented by a seriesof black dots on the diagram. A trigger event 1130 (indicated by dashedline) occurs when the received GPS information falls within thegeographic area 1136. As indicated, trigger event 1130 occurs at a timet₂. Acoustic information is collected while communication device 1131 iswithin geographic area 1136. The event end 1132 (indicated by dashedline) occurs when communication device 1131 (from received GPSinformation) falls outside the geographic area 1136. Event end 1132occurs at a time t₃. As mentioned hereinabove, received acousticinformation is stored in a buffer. The acoustic information 1138corresponding to the time period between trigger event 1130 and theevent end 1132 is collected for sending to a database. Acousticinformation 1138 can be moved from the buffer to more permanent memoryto be uploaded to the database at an appropriate time.

The acoustic information 1138 can be stored in memory of thecommunication device in it's entirety or converted to a more compactform, modeled, characterized, and provided with metadata depending onthe collection needs. In at least one exemplary embodiment, acousticinformation 1138 is provided with metadata that includes timeinformation and geographic location information. The time information isprovided from clock circuitry in the communication device. In thedisclosed embodiment, geographic location information provided throughthe GPS receiver 1134 of the communication device is provided indiscrete time intervals. Linear interpolation or other interpolationmethodologies can be use to estimate geographic location during timeperiods between received data points from the GPS receiver 1134.

FIG. 14 is a diagram illustrating the use of time as a trigger eventconfigured to collect acoustic information. A communication device 1140includes clock for providing time information. One or more times arestored in communication device 1140 or communication device 1140 hasaccess to times for initiating a trigger event based on time. A storedtime includes an event trigger time (or start time configured to collectacoustic information) and an event end (or end time configured tocollect acoustic information). Communication device 1140 compares thecurrent time with the one or more stored times. A trigger eventconfigured to collect acoustic information occurs when the current timecorresponds to a stored event trigger time.

An event trigger 1144 configured to collect acoustic information occurswhen the clock of a communication device 1140 corresponds to a time t₂.An event end 1146 ends the collection of acoustic information and occurswhen the clock corresponds to a time t₃. As disclosed hereinabove, timest₂ and t₃ are stored in communication device 1140 for triggering thecollection of acoustic information. In at least one exemplaryembodiment, acoustic information 1148 is transferred from a buffer to amore permanent memory of communication device 1140 for uploading at anappropriate time.

In at least one exemplary embodiment, GPS receiver 1142 periodicallyprovides information on the location of a communication device 1140. Theperiodicity of the received GPS information is represented by a seriesof black dots on the diagram. Linear interpolation or otherinterpolation methodologies can be use to estimate geographic locationduring time periods between received data points from the GPS receiver.GPS information and time information corresponding to acousticinformation 1148 is provided along with other metadata. The timeinformation is provided by the clock in communication device 1140. Theacoustic information 1138 can be stored in memory of the communicationdevice in it's entirety or converted to a more compact form, modeled,characterized, and provided with metadata depending on the collectionneeds.

FIG. 15 is a diagram illustrating the detection of a sound signature asa trigger event configured to collect acoustic information in accordancewith at least one exemplary embodiment. A communication device 1150receives acoustic information from a microphone. Communication device1150 has in memory one or more sound signatures or has access to one ormore sound signatures for comparison to acoustic information received bythe microphone. The acoustic information is continuously assessed forsimilarities to the sound signatures. A trigger event occurs when theacoustic information is found to be similar to a sound signature. Thetrigger event initiates the collection of acoustic information for beingprovided to a database.

In at least one exemplary embodiment, acoustic information is stored ina buffer of communication device 1150 as a digital sound signal. Thedigital sound signal can be broken up into frames of information. Forexample, a frame is defined as sound information corresponding to a 20millisecond sample at a sampling frequency of f_(s)=8000 Hz which yields160 samples per frame. Each sample is quantized to a number representedby the bit resolution of the A/D converter (e.g. 13 bits or 24 bits) ofcommunication device 1150. For example, a number generated by the A/Dconverter may represent a voltage corresponding to a voltage output ofthe microphone at the time of a sample.

Although not drawn to scale a series of frames is shown versus time.Each frame has digital sound information associated with it thatcorresponds to the acoustic information captured by the microphone. Thedigital sound information is processed to extract features related tothe received sound. As disclosed hereinabove, the features can beFourier series coefficients (FFT based) that represent a frequencydecomposition of the frame; the features can be mel-cepstral or LPCcoefficients that represent a spectral envelope decomposition. Thefeatures can be DCT, KLT, PCA, or any other feature set. Notably, thefeatures provide an efficient form of data compression to reduce thedimensionality of the input space. For example, instead of using all 160samples (of a single 20 millisecond frame sampling at f_(s)=8000 Hz) themel-cepstral only requires 10-14 samples (16 bits/sample) to representthe same frame.

Associated with each frame of acoustic information are extractedfeatures 1164. In at least one exemplary embodiment, communicationdevice 1150 extracts the features from each frame of acousticinformation and compares the features one or more sound signature occursframe by frame. As shown a frame of extracted features 1156 of theacoustic information is compared against the GMMs of the soundsignatures. In general, extracted features 1164 of a frame of acousticinformation is mapped to parameters (means, covars, weights) of thesound signatures to determine which GMM has the highest likelihood ofrepresenting the features of the frame of acoustic information. Thecomparison yields a probability (for example between 0 and 1) of howwell it matches each GMM of a sound signature. A trigger event 1158occurs if the comparison meets the criteria, for example a probabilitygreater than 0.8. It should be noted that the criteria can be varied andis selected to best capture or collect acoustic information that aresimilar or related to the sound signatures it is being compared to. Thecriteria set too high may yield very little collected acousticinformation while setting the criteria too low may collect a substantialamount acoustic information unrelated to the sound signatures. In thisexample, setting the probability threshold to a level of 0.8 wouldtrigger the collection of information if the acoustic informationcorresponded to the sound signature with an 80% likelihood. In at leastone exemplary embodiment, the event would end when the criteria is notmet 1160 as indicated on the diagram.

In at least one exemplary embodiment, collected acoustic information1162 comprises more than the acoustic information that resides betweentrigger event 1158 and the time that the acoustic information no longermeets criteria 1160. Once it has been established that the acousticinformation meets the criteria (above probability threshold) and shouldbe collected, it is desirable to ensure that the entire sound iscaptured (not just the portion that meets the criteria). Thus, acousticinformation preceding trigger event 1158 is collected as well asacoustic information after the criteria is not met. The amount of addedtime of the collected acoustic information pre and post can be of afixed amount or based off of other methodologies for determining howmuch acoustic information to collect. In general, the time span ofacoustic information to be collected is not an issue because theacoustic information is stored in a buffer and the time span of acousticinformation is merely transferred to longer term memory in communicationdevice 1150.

As disclosed hereinabove, metadata including time information andgeographic information corresponding respectively to when and where theacoustic information was received. The metadata can attached to theacoustic information. A clock in communication device 1150 provides thetime information. A GPS receiver 1152 provides periodic geographiclocation information and is indicated as a series of black dots in thefigure. Interpolation can be used to estimate location between datapoints provided by GPS receiver 1152.

Alternately, voice activation could be used as a trigger event. Forexample, voice recognition circuitry in cell phone 1150 could detectwords spoken by the user that initiate an action. Similarly, thedetection by cell phone 1150 of a recognized phrase or group of words(in different languages) by another could initiate the trigger event.For example, scream for help or dial 911. Cell phone 1150 could send theacoustic information, time, and location to an appropriate entity inresponse to the trigger event.

FIG. 15a illustrates triggering using an SPL value in a frequency band.1500 illustrates an acoustic signal, which can be analyzed in varioustime increments (e.g., from tb1 to tb2). The analysis can do a spectralrepresentation (e.g., FFT) 1500A, which provides spectral information(e.g., Power spectral density PSD) that can be used as an initialtrigger event. For example instead of analyzing signals for SSDcontinuously, which can be resource intensive, one can look for peaks invarious SPL frequency bands. If the peak exceeds a threshold value, SSDanalysis can be triggered to identify the signal that resulted in theSPL peak. One can look at a single or multiple frequency bands, whereeach can have its own threshold value. For example 1510 illustrates anacoustic temporal signal that has a pattern. Analysis between timeincrements tb3 to tb4 results in a spectral representation 1510A. Apreviously stored threshold value 1540 (e.g., 3 dB above the noisefloor) in the frequency band Fmin (1520) to Fmax (1530) triggers afurther SSD analysis with a time increment tc1 to tc2.

FIG. 15b illustrates a flow chart describing a triggering event usingperiodic signals. In accordance with another exemplary embodiment, andinput acoustic signal 1547 can be analyzed 1551, to determine if thereis a periodic signal 1553. If there is a periodic signal then theacoustic data needed for sonic signature detection (SSD) analysis can beextracted from a data buffer 1557. The extracted data can be used togenerate sonic signature parameters (SSP) 1561 for example coefficientsfor a Gaussian mixture model. The SSPs calculated can be compared withstored values 1563 to determine if identification of the signal can beobtained. If the signal is identified 1565 then a stored actionassociated with the signal can be enacted 1567. For example if a firealarm is identified, then attenuation can be reduced (e.g., if using anactive inflation management system). If the signal has not beenidentified then notification can be sent 1569 identifying the signal asa new or unknown signal.

FIGS. 15c-15k illustrates various spectrogram signatures that illustratethe use of spectrograms for periodic detection in accordance with atleast one exemplary embodiment. One of the possible methods ofidentifying periodic signals is the use of spectrograms. For example adetector can measure acoustic signals over time. The signals can bebroken into a spectrogram, where a certain time fragment is convertedinto spectral information (e.g., PSD) and is associated with a column ofpixels in the spectrogram. The columns values in the columns can benormalized so that each column of the spectrogram has the same max andmin value, where the max and min range can be broken into discretevalues and the column values reassigned to their respective discretevalues. The normalized (or un-normalized) spectrogram over time can beanalyzed to look for peaks above a threshold level. An analysis regioncan then be centered about the peak and re-centered taking into accountadditional peaks in the analysis region to derive a center of gravity ofthe peak regions (e.g., centerline). If it is determined or expectedthat the signals are speech, then temporal stretching or reduction canbe enacted within an analysis region to fit signal levels above athreshold within the full extent of the analysis region (not shown).Then a correlation value can be taken between neighboring analysisregions to determine the likelihood of repeating signals.

For example FIG. 15c illustrates the word “fire” spoken by a firstspeaker three times in succession (e.g., Analysis region R1, R2, andR3). The distinctive pattern is evident. Upon centering/recentering, thecorrelation between neighboring analysis regions in FIG. 15c would begood (e.g., >0.6). FIG. 15d illustrates three different words spoken byspeaker 1 “fire”, “wire”, and “mire” associated with analysis regionsR4, R5, and R6 respectively. Upon centering one can determine thedifference in the patterns and one would get a lower correlation value(e.g., <0.6) indicating a non-repetitive signal.

FIGS. 15e and 15f analogous with FIGS. 15c and 15d , but for a secondspeaker. One can note the similarity in the pattern between the word“fire” in FIG. 15e and the word “fire” in FIG. 15c . FIGS. 15g and 15hillustrate spectrograms of three various words “fire”, “help”, and“police” from speaker 1 and speaker 2 respectively. One can notice someof the similarity between common words regardless of the speaker.

FIG. 15i illustrates spectrograms of a frequency sweep alarm, FIG. 15jillustrates spectrograms of a car horn, and FIG. 15k illustrates thespectrogram of a fire alarm. Note that a very long temporal signal canresult in the necessity of increasing the analysis region width. One cansee the difference between signals between FIGS. 15i, 15j , and 15 k.

FIGS. 16a-16c are related diagrams illustrating the use of sensor dataas a trigger event configured to collect acoustic information inaccordance with at least one exemplary embodiment. Sensors and sensordata are useful in determining a trigger event configured to collectacoustic information. Examples of sensor data are acceleration andvelocity, temperature, atmospheric measurements (barometric pressure,wind speed, moisture levels, etc.), odor (smell), chemical detection,biological information (heart rate, blood pressure, glucose level, etc.)to name but a few. Velocity and acceleration will be used to illustratea trigger event but the other sensor data could be used in a similarfashion to initiate collection of acoustic material.

Referring to FIG. 16a , a graph of acceleration versus time is shownthat is calculated by communication device 1170. Similarly, referring toFIG. 16b , a graph of velocity versus time is plotted in FIG. 16b .FIGS. 16a and 16b illustrates a trigger event 1172 configured to collectacoustic information when an acceleration threshold 1174 is exceeded. Inat least one exemplary embodiment, information on velocity andacceleration can be provided by an accelerometer 1178 in communicationwith device 1170 or residing within communication device 1170. Forexample, the user of communication device 1170 is on a roller coasterthat rapidly accelerates. The acceleration can be calculated from therate of change in velocity measured by accelerometer 1178. Should thecalculated acceleration exceed acceleration threshold 1174 acousticinformation is collected. Note that such an arrangement can be used in ablack box arrangement. For example in a car accident, uponidentification of acceleration and/or deceleration levels that exceedthreshold levels (e.g., 2 gs) the sound recording can be started andstored. It can later be accessed to provide clues for the accident.

In at least one exemplary embodiment, trigger event 1172 initiatescollection of the acoustic information during a time period (t₂-t₃) inwhich the trigger event occurs. The collection of acoustic informationis not limited to time period t₂-t₃ and can be adjusted based on whatthe collection need. In at least one exemplary embodiment, acousticinformation is stored in a buffer and thus can be retrieved for a timespan greater than the period where trigger event 1172 occurred.Conversely, acoustic information could be collected for a time less thant₂-t₃, for example, during the period where acceleration exceedsacceleration threshold 1174.

Once triggered, the acoustic information as shown in FIG. 16c , duringtime period t₂-t₃ is collected for sending to a database. The acousticinformation can be moved from the buffer to more long term memory in thecommunication device in it's entirety or converted to a more compactform, modeled, characterized, and provided with metadata depending onthe collection needs. The collected acoustic information could also besent immediately from communication device 1170 to the database.Metadata aids in identifying and describing the acoustic information andtrigger event. The metadata can attached with the collected acousticinformation and includes time information and geographical locationinformation. In general, a clock in communication device 1170 providesthe time information or time stamp and a GPS receiver 1176 providesgeographical location information or geocode where the acousticinformation was received.

In general, multiple trigger events can be used in an AND or ORcombination. An example of using both acceleration and biological sensorinformation illustrates a potential life saving application. Abiological sensor is coupled to a user for monitoring vital functions ofthe user of communication device 1170. The biological sensor is incommunication with communication device 1170 that also includesaccelerometer 1178. The user is traveling in a vehicle that undergoesrapid deceleration that ultimately results in an accident. Accelerationthreshold 1174 (absolute value) is exceeded and the acoustic informationis analyzed by cell phone 1170 indicating a serious accident hasoccurred. The analysis could include sound signature detection thatcalculates a high probability that it is a high impact accident(including the accelerometer measurements). The biological sensorprovides information on the health of the user. Communication device1170 could automatically send the information (time, location,parameters of the accident) as well as pertinent personal medicalinformation (medical conditions, medicines, allergies, etc.) to thepolice, fire department, and hospital resulting in saving minutes oftime that could save the user's life. A further feature could be that atrigger event or trigger events could initiate pictures being taken or avideo being taken. Moreover, if circular buffer was being employed forstoring visual information it would be possible to provide visualinformation preceding the trigger event and after the trigger event.

As disclosed, a trigger event can be enabled by the user of the deviceor remotely. For example, a parent may want to enable a trigger eventthat sends acoustic information to the parent if the child's phonerecognizes a warning signal (example gun shot), a sound pressure levelis exceeded, or a geographic location is entered. The parent could enterthe trigger event into the child's phone or enter it remotely to providenotification if certain events occurred.

FIG. 17 is a block diagram illustrating downloading from a catalogue ofsound signatures 1210 in accordance with at least one exemplaryembodiment. As mentioned previously, there is tremendous benefit tohaving a database of sounds collected across the globe. For example, thedatabase of sounds represents an accessible history of the audibleenvironmental changes recorded for posterity. Scientists and historianscould use the information for their studies. A system was disclosedherein that uses sound signature to identify similar sounds forpersonalized sound management applications. Similarly, the sounddatabase would have applications to business and governments. Forexample, sound effects for movies or for analysis of noise levels inurban versus city environments.

Website 1202 can provide one or more web pages for individuals,governments, business, scientists, etc. for providing sound signatureinformation for applications that provide utility, service, or product1204 as described herein. A user or entity performs a search 1206 ofsound database 1208. Catalogue of sound signatures 1210 are organized tooptimize searching of this vast database. The sound signatureinformation is referenced, correlated, cross-correlated, technicallymodeled, geocoded, time stamped, context, content related, applicationrelated, and others in hierarchy that allows sound signatures to beidentified and found. The user can search 1206 until the appropriateitem sound signature or sound pressure level is found.

Once found, sound content 1212 of one or more sound signatures as wellas all information associated with the sound signatures is provided orlinked to a web page 1214. A playback environment 1216 is provided forplaying the sound signatures. The user may download the sound signatureand information for use in their application. In at least one exemplaryembodiment, depending on the application a fee may be associated withthe downloading of sound signatures. For example, a movie studio wantinga collection of hundreds or thousands of gun shots and explosions wouldhave to pay for this database of sounds.

FIG. 18 is a block diagram illustrating an application where providingsound is informative and enhances a searching experience in accordancewith at least one exemplary embodiment. Internet searches of maps bringup many different images of an area search. It can be topographical,satellite image, street map, or other types of map view. The one missingpiece in many different types of searching is providing auditoryfeedback. For example, a search of a street map might visually tell youwhat the street looks like but it is a static image. Imagine if youcould hear as well as see what the street looks like at different timesof day. An audio experience brings an entire new dimension to this typeof use.

A user can perform a search 1302 on the internet. For example, the useris traveling and is located in an unfamiliar area that he or she doesnot know well. It should be noted that the concept disclosed herein canbe applied, in general, to a search and the specific example disclosedis for illustration purposes only. Search 1302 is for restaurants thatare within walking distance from the hotel where the user is staying.The user has to criteria business proposal to be presented the next day.Search 1302 displays map 1304. This is indicated in a screen shot 1312.In at least one exemplary embodiment, the screen shot provides a mapwith restaurant locations. The restaurants can be listed and categorizedby restaurant type to aid the user in their search.

In at least one exemplary embodiment, search 1302 comprises a search ofsound database 1310. Sound database 1310 has a search hierarchy of whichone is grouping restaurant information and another is locationsupporting an application such as map 1304. In at least one exemplaryembodiment, when the user places the cursor over a specific restaurantmore information is displayed. The information can be visual (picture,text) and audio from database 1310. As disclosed herein, soundinformation of all types is automatically and manually provided todatabase 1310. In at least one of the exemplary embodiments, some of theinformation in database 1310 relates to restaurants in the search areathat has been amassed from opening to closing of each restaurant, everyday of the year. Examples of information received from database 1310 aresounds of the restaurant, sound pressure levels in the restaurant, userreviews of the restaurant, sounds around the area of the restaurant,location, and menu. Thus, the user could listen to reviews by actualcustomers, find out signature dishes of the restaurant, or determine ifthe area and restaurant is too loud or noisy. For example, the userwants a quiet dinner. Information provided indicates that the restauranthas low sound pressure levels indicating a quiet atmosphere. The userchecks the time period associated with the week of his/her stay andfinds out that the noise levels jump up because of the influx of collegestudents on break during this time of the year. Similarly, the usercould find out that certain times are substantially more quiet (exampleprior to and after happy hour) and arrange his/her schedule to go duringa quiet period as indicated by the information. Thus, user utility andexperience is greatly enhanced by providing audio information directedto the subject of the search.

FIG. 19 is a block diagram illustrating an application ofthree-dimensional acoustical mapping in accordance with at least oneexemplary embodiment. As described hereinabove, each sound signature hasan associated time stamp and geocode. The geocode gives a location ofeach sound signature in three dimensional space. Thus, a measurement onthe first floor of a building will have different coordinates thansomeone directly above on the second floor. Thus, a group of soundsignatures can produce a three dimensional acoustic map of an area. Anexample would be sound pressure level variations in and around abuilding. Another example would be the types of sounds that are heard(e.g. horns, industrial noise, office chatter, elevator sound, etc.).

An example illustrating three dimensional acoustic mapping is of abuilding 1402. It should be noted that the concept disclosed herein canbe applied to any type of three dimensional space and the example is forillustration purposes only. Sound signatures around the area of building1402 have been collected automatically and manually by people who work,visit, or live in the area throughout the year. The sound signatures,sound pressure level measurements, and other acoustic information arestored in sound database 1408. A search provides information fromdatabase 1408 around building 1402, inside building 1402, at differenttimes of the day, and at different times of the year (using time stampinformation). An acoustic map program 1406 uses the acoustic informationfor building the sound map 1410 of the building interior and thesurrounding area outside building 1402. Sound map 1410 could bedisplayed at different times of the day, over weeks, years, andetcetera. Visual maps can also be integrated with the 3D acoustic mapproviding even more information.

For example, a business wants to relocate to an area that they are notfamiliar with. The office they are interested in leasing is on thesecond floor of a building having a manufacturing facility on the firstfloor. Owners of the building have told them that the building is wellinsulated for noise. An acoustic map of the area could be generatedusing the sound signatures from database 1408 to investigate noiselevels. The sound pressure level measurements associated with the soundsignatures would be used extensively in the sound map. In this example,the sound map generated did indeed establish that the sound from themanufacturing facility was nominal but the map indicated that one sideof the building suffers from substantially high wind noise duringcertain times of the year while another side receives substantial noisefrom an adjacent building. Furthermore, analysis of recordings usingsonic signature detection identified train noise and motorcycle noiseduring certain parts of the day. Thus, the decision of the business wasnot to lease the building. Sound maps of other buildings were generatedand used in the decision process to identify the best solution.

FIG. 20 is a block diagram illustrating an application for automaticallyproviding emergency information in accordance with at least oneexemplary embodiment. In general, when a life threatening incidentoccurs time is of the essence. A communication device 1502 such as acell phone, earpiece, personal digital assistant, smart phone, or laptopcomputer adapted for running personalized sound management software asdisclosed in FIG. 9 for an earpiece can be used to detect a dangeroussituation that should be reported to the proper authorities.

A communication device 1502 runs personalized sound managementapplications 1504. In particular, always on recording 1508, storesacoustic information in a buffer and continuously analyzes soundprovided by the microphone of device 1502 as disclosed hereinabove. Theanalysis includes comparing the acoustic information to sound signaturesthat are stored in memory of device 1502 or through access to a libraryof sound signatures. The analysis is performed in a step of soundsignature detection 1510. In this example, a subset of the soundsignatures relate to emergency sounds.

An emergency sound is one of significance to authorities such as thepolice, hospital, fire department, or other authorities. Examples ofemergency sounds are a gun shot, a car crash, a cry for help, anexplosion, or other sound that may be life threatening or require animmediate response. The step of sound signature detection 1510 analyzessound information received by the microphone of device 1502. If a normalsound signature is detected, a response is provided based on thepersonalization of applications 1504.

An emergency sound 1506 is received by the microphone of communicationdevice 1502. Emergency sound 1506 is stored in a buffer and analyzed.The step of sound signature detection 1510 finds the sound similar toone or more sound signature emergency sounds. The sound can be put intoa sound category 1514 that selects an appropriate response for theemergency sound. For example, a gun shot is detected, gun shots are in asound category 1514 that provides a response for creating andautomatically sending a report to 911. In at least one exemplaryembodiment, information such as the type of gun shot, the direction ofthe gun shot, the distance from the communication device, when, andwhere the sound occurred could be supplied in a step of provideinformation 1520. Communication device 1502 would automaticallycommunicate and provide information to the appropriate authorities. Inthe methodology described herein, multiple people at different locationscould automatically provide information 1520 to the police. Acousticmapping could be used to by the police to further determine the severityof the situation and provide a response that can save people's lives andprotect the authorities that would be in harm's way.

FIG. 21 is a block diagram illustrating an application for detecting aburglary, intrusion, or serious situation in a building or home inaccordance with at least one exemplary embodiment. A communicationdevice 1602 such as a cell phone, earpiece, personal digital assistant,smart phone, or laptop computer adapted for running personalized soundmanagement software as disclosed in FIG. 9 for an earpiece can be usedto detect a burglary, intrusion, or other serious situation in the homethat should be reported to the proper authorities. Similarly, a deviceusing sound signature detection could be built for the specific purposeof monitoring a home.

A communication device 1602 runs personalized sound managementapplications 1604. In particular, always on recording 1608, storesacoustic information in a buffer and continuously analyzes soundprovided by the microphone of device 1602. The analysis includescomparing the acoustic information to sound signatures that are storedin memory of device 1602 or through access to a library of soundsignatures. The analysis is performed in a step of sound signaturedetection 1610. In this example, a subset of the sound signatures relateto sounds corresponding to home intrusion or other serious event such asa fire.

A burglary sound 1604 is used as an example of device 1602 detecting andresponding to the sound. Examples of a burglary sound are the sound ofshattering glass to gain entry, sounds of someone trying to open windows(window rattling), or the rapid opening and closing of cabinet doors toname but a few. The step of sound signature detection 1610 analyzessound information received by the microphone of device 1602. If a normalsound signature is detected, a response is provided based on thepersonalization of applications 1604.

In the example, a burglary sound 1606 is received by the microphone ofdevice 1602. Burglary sound 1606 is stored in a buffer and analyzed. Thestep of sound signature detection 1610 finds the sound similar to one ormore sound signature burglary sounds. The sound can be put into a soundcategory 1614 that selects an appropriate response for the burglarysound 1604. For example, a sound similar to a window breaking isdetected by device 1602 which also knows that the geographic location ofdevice 1602 corresponds to the home of the device owner. The sound ofwindow breaking is in a sound category 1614 that provides a response forcreating and automatically sending a report to the local security in thehousing complex. In at least one exemplary embodiment, information suchas the type of window breakage, the direction of the window breakage,the distance from of the device 1602 from the window breakage, when, andwhere the window breakage occurred could be supplied in a step ofprovide information 1620. Communication device 1602 would automaticallycommunicate and provide information to the appropriate authorities. Inthis example, local security to the complex would be notified and coulddrive out and determine if something is amiss. Similarly, the policecould be contacted and they could send someone in the vicinity to checkthe property out.

FIG. 22 is a diagram illustrating a website 1706 including a personalwebpage 1710 for socialization having an audio locker 1712 in accordancewith at least one exemplary embodiment. Participants in using devicesfor sound capture will also be capable of tagging and cataloging thesesounds based on their preferences for the purpose of socialcollaboration. Pre-established audio lockers will be available throughan online interface with customizable lockers definable by thecommunity. For example, website 1706 provides an environment forsocialization. Users of website 1706 can have a personal webpage 1710that includes an audio locker 1712 that is secure and under usercontrol. Other website users will not have access to audio locker 1712which is password protected. Audio locker 1712 is for personal use instoring personal audio information, from communication device 1702.

Communication device 1702 can be used to create an audio biography ofthe user. In at least one exemplary embodiment, device 1702 receives andstores audio information that is an audio record of the user's day. Inan upload step 1708, device 1702 is connected to the user audio locker1712 on website 1706 or though a user system (e.g. personal computer)for uploading audio information stored of the day's events. The user canedit or create clips of the audio information through the playbackenvironment on the website. Audio information that the user desires toshare with the public can be posted on the user personal webpage 1710.Global users 1704 of website 1706 can access the personal web page ofthe user and listen to the audio biographical information provided bythe user through playback environment 1714. An example of personalwebpage 1710 audio contents provided by the user for socialization isillustrated below:

On this day . . . .

-   -   Important events—Sporting, Party, Truck Show, Parade, Skeet        shoot

-   My home—morning, dinner time, weekends

-   My street—my yard, street corner

-   My favorite places—restaurant, park

-   My noisiest places—local construction, subway/train station, airport

-   My entertainment—street events, concerts, video arcade, gun range

-   My commute

-   My last adventure

-   My favorite sounds

-   My least favorite sounds

-   You have to hear this

These content segments will allow for the user to publish throughapplications to their online social sites, start discussions in open andclosed communities, and overlay their geocoded and time stamped soundson 3^(rd) party mapping programs. Collected sounds could also belinkable into recommendation and rating engines (XYZ restaurant at 6:30on Wednesday night) that provide public input on various venues. Thus,the collection of biographical audio information could have significantsocial impact in linking people to one another, creating topics ofdiscussion that is based on audio segments, and in general enhance theexperience of using a socialization website.

In an alternate exemplary embodiment, website 1706 providessocialization between grandparents and grandchildren allowinginteractive or delayed communication. For example, personal webpage 1710is associated with a specific grandparents and grandchildren. Personalwebpage 1710 is a secure user space with an owner specifying the users(grandparents and grandchildren) having access. More than one audiolocker 1712 can exist within personal webpage 1710 or audio locker 1712can be partitioned with more than one secure and specific access areas.

Audio locker 1712 is a safe storage area for audio information as wellas other information such as videos or pictures. Grandparents andgrandchildren can use it as a repository for posting audio informationfor fun or communication allowing for continuous interaction.Communication device 1702 can be used to provide audio information viaan upload 1708 to audio locker 1712. Audio locker 1712 can be a conduitfor children to send audio messages to their grandparents that they maynormally have difficulty asking. For example, grandchildren can place anaudio wish list of presents they want for their birthday in audio locker1712 or the audio wish list can be placed in a secure area of personalwebpage 1710 for review by the corresponding grandparent.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

What is claimed is:
 1. An electronic device configured to collectacoustic information, comprising: at least one or more processors; and anon-transitory computer readable medium communicatively coupled to theat least one or more processors, the execution of the instruction by theat least one or more processors causing the at least one or moreprocessors to perform operations comprising: collecting acoustic data bya microphone communicatively coupled to a mobile device; analyzing theacoustic data for a sound signature; tagging the sound signature withmetadata; sending the sound signature with metadata to an acousticdatabase; associating sounds within the acoustic data with information;presenting the information on a map on the mobile device; and accessingat least audio from the acoustic database when a cursor is placed over aspecific location on the map corresponding to captured ambient soundsfrom a geographic location and wherein acoustic information can beretrieved corresponding to the geographic location and for differentselected periods of time.
 2. The electronic device of claim 1, whereinacoustic information can be retrieved corresponding to the geographiclocation and for different selected times of a day or for differentselected times of a year.
 3. The electronic device of claim 1, furthercomprising audibly presenting the audio from the acoustic database whenplacing the cursor over the specific location on the map correspondingto the geographic location and representative of the captured ambientsound collected at the specific location.
 4. The electronic device ofclaim 1, wherein the information on the map is visual information andwherein the electronic device is further configured for searching audiocontent associated with the map related to the visual information wherethe searching includes downloading or streaming media to the mobiledevice for analysis of the audio content.
 5. The electronic device ofclaim 1, where the map comprises at least a picture or text and wherethe metadata includes at least time information or location informationwhen the acoustic data was collected.
 6. The electronic device of claim1, further comprising searching the acoustic database for acousticinformation related to a topic, and providing information posted withthe location information corresponding to the map related to the topicincluding time information.
 7. The electronic device of claim 1, furthercomprising: storing the acoustic data as an audio record of at least aportion of a user's day; uploading events in the audio record determinedin the user's day; and sharing the audio record of events for socialnetworking.
 8. The electronic device of claim 1, further comprisingstoring the acoustic data as an audio segment and linking discussiontopics on the audio segment across social websites.
 9. The electronicdevice of claim 1, wherein the step of collecting comprisesautomatically and continuously capturing acoustic data responsive to thetrigger event by the microphone communicatively coupled to the mobiledevice and wherein the step of presenting comprises presenting a topicassociated with the acoustic data and the map.
 10. The electronic deviceof claim 1, wherein the different selected periods of time correspond tosample frames of information.
 11. The electronic device of claim 1,wherein the one or more processors further to perform operationscomprising at least one of storing the acoustic data or metadatatemporarily or storing the acoustic data or metadata permanently.
 12. Anelectronic device configured to collect acoustic information,comprising: at least one or more processors; a location moduleoperatively coupled to the one or more processors and configured todetermine a location of the electronic device; a transducer operativelycoupled to the one or more processors and configured to receive acousticdata; and a non-transitory computer readable medium communicativelycoupled to the at least one or more processors, the execution of theinstruction by the at least one or more processors causing the at leastone or more processors to perform operations comprising: collectingacoustic data via the transducer; analyzing the acoustic data for asound signature; tagging the sound signature with metadata; sending thesound signature with metadata to an acoustic database; associatingsounds within the acoustic data with locations on a map; and accessingat least audio from the acoustic database when a cursor is placed over aspecific location on the map corresponding to captured ambient soundsfrom a geographic location and wherein acoustic information can beretrieved corresponding to the geographic location and for differentselected periods of time.
 13. The electronic device of claim 12, whereinthe transducer is a microphone.
 14. The electronic device of claim 12,wherein the execution of the instruction by the at least one or moreprocessors further causes the at least one or more processors to performoperations comprising searching for content in the map related to thesounds responsive to voice activation of spoken words.
 15. Theelectronic device of claim 12, wherein the execution of the instructionby the at least one or more processors further causes the at least oneor more processors to perform operations comprising searching forcontent from an acoustic map of the world for a selected period of timeof a day.
 16. The electronic device of claim 12, wherein the executionof the instruction by the at least one or more processors further causesthe at least one or more processors to perform operations comprisingstoring acoustic information in a secure sound locker.
 17. Theelectronic device of claim 12, wherein the execution of the instructionby the at least one or more processors further causes the at least oneor more processors to perform operations comprising linking topics onthe map across social websites including an overlay of geocoded and timestamped sounds on the maps representative of the captured ambient soundsfrom the geographical locations.
 18. A method of collecting acousticinformation, comprising: collecting, by a server, acoustic data via amicrophone communicatively coupled to a mobile device; analyzing soundswithin the acoustic data for a sound signature; tagging the soundsignature with metadata; storing the sound signature with metadata to anacoustic database; associating sounds within the acoustic data withinformation on a map; and providing access to at least audio from theacoustic database responsive to a signal corresponding to when a cursoris placed over a specific location on the map corresponding to capturedambient sounds from a geographic location for different selected periodsof time.
 19. The method of claim 18, wherein the different selectedperiods of time correspond to one or more of a selected time of a day, aselected time of a year, or a selected frame of information.
 20. Themethod of claim 18, wherein the map comprises a real-timethree-dimensional acoustic map that is produced or updated usingsearchable sound content including real-time acoustically mappedinformation from the geographical location.